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OBJECTIVE: 

Tile  purpose  of  this  research  wa®  ^  b^related'tiflater  fli^t'sinwbon  awarene^ (S/0  ^  ^ 

develop  new  tests  of  cogmttve  abtltfes  that  ™ “  b '  *“es  of  cognilve  ability,  and  to  give  the  new 

^S^a,dsofoecf,aL„s.eqniringSAwi„ 

be  enhanced. 

from  a  tbeoretioai  perspective,  t.n^earei,  was  designed 

abilities  required  to  gam  and  maintam  flig  ,  fg P  nt  and  rt  pilot 

SA  construct  In  “  ns  on  S  velopmen.  and  «  of  internal  and  external 

WM,  LTWM,  and  flight  SA  abides  a nd  to : “  on  ,sPwere  supposed  t0  be  accomplished  during 

::Sa“ 

meant  the  additional  goals  could  not  be  addressed. 

Background 

✓-a  winricinCT  mpmorv  tWM)  defined  as  a  limited,  temporary  store  for  processing 

Working  Memory  C“P“cl^  ev  &  Hitch  ?974)  has  been  studied  extensively  in  various  cognitive  tasks, 
and  storing  information  (Baddeley  &  Hitciu  *  exnlained  bv  WM  capacity.  For  example, 

Individual  differences  n^co^mve^ask  pCTfoiman^^^  ^  comprehension.  Once  an  individual’s 

Just  and  Carpenter  (1992)  P  P  processing  and  storage  hinders  the  ability  to  use  and  retain 

WM  capacity  is  reached  the  ^kof  available  pro  g  ^  ^  mformatlon;  resuitillg  in 

new  infonnation,  as  we  as  m  erm  sueaests  a  role  of  WM  capacity  in  performing  additional 

decreased  comprehension,  u  ler  r  Mivake  1996),  the  ability  to  follow  complex 

cognitive  tasks  such 1  ^  sPa^a  “9  (and  computer  problem  solving  (e.g.,  Anderson  &  Jeffries, 

directions  (Engle,  Cara  >  k  Clawson  1992'  Doane,  Sohn,  McNamara,  &  Adams,  2000; 

WM  is  thought  to  reside  in  short  term  memory 

believedto  have  ,  fixed  capacity,  although  individuals  differ  in  flieir  capacnes;  and  an  mdrvidnal  s  WM 
capacity  remains  stable  over  time  (e.g.,  Baddeley  &  Hitch,  19  ). 

_  , .  w  t  twtvI  k  n  theorv  of  a  memory  process  that  explains  how  individuals  can 

Long-Term  Working  Memory.  oi  i  two  chunks  (Miller,  1956).  In  LTWM 

extend  WM  capacity  well  beyond  tl  p  P  P  .  thought  to  increase  individual  ability  to 

theory,  domain-specific  knowledge  g^ijy^^cessMe  retrieval  structures 

encode  infonnation  into  ong  cun  mei  retrieval  structure  as  an  organization  of  meaningful  data 

efficiently  Ericsson  and  Kmtsch  and  retrieve  information  from  LTM 

’“mplTe d™«MhSi»8  indices  te<!uire<1  “  re,rieVe  ,l"  stoa,i°n-sp“,r,C  The 


indices  change  dynamically  as  a  fund, on  of  the  task  at  hand  and  the  individual’s  expertise  for  that 
particular  task. 

Situation  Awareness.  The  term  situation  awareness  (SA) .  has  been  u^lighte as  a 
military  aviation  domains  because  of  its  prominent  role  in  flight  opei.trais.  Altnougn 

leading  cause  in  military  aviation  ^c^ensus  regarding  the  definition  of  SA  and  little 

Salas,  Prince,  Baker,  &  Shreshta  1995>' “"ft?  “  &  Sontad  1W).^ Because  of  its  lack  of  definition, 
empirical  evidence  .0  support  a  explanation  of  human 

(1995, 2000)  information  processing  view.  In  En  y •  bjU  comprehend  the 

lSA  involves  the 

^rF*i"nds“  fsJKSSy  to  use  the  Sima, ion  mode,  created  in  Level  2 ,0  predict 
the  future  status  of  the  system. 

APPROACH: 

The  focus  of  this  research  was  on 

information,  the  mechanisms  associa  e  nractice  An  information  processing  approach  was 

^ =:^m,!:o*ing^sk„,s 

influence  pilot  flight  SA  performance  as  a  function  of  expertise. 

The  firs,  year  was  spen.  refining  measures of  WM 

novice  and  expert  pilots  in  an  expetimtm.  deigned  ,o«  ft  n fd.  md  instructor 

pilotWh^LTWM^and  flight  SAabiimes^n^tiie  development  and  test  of  internal  and  external  validity  for 
three  new  measures  of  flight  SA  ability,  respectively  -  were  not  addiessed. 

description  of  each  of  the  expeiiments. 

Caipenter,  1980)  and  spatia^ WM  (^iah  &  Miyake^l996).  In  this  task^ jrilots  were 
indicator  displays  posit, oned  m  diffeient  flight  orientations  (see  Hgu  ^  ^  ^  P  J  ^  aircraft  was 

□  ^  -^i 
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Figure  1.  (a)  Pitched  down  attitude  indicator,  (b)  Pitched  up 
attitude  indicator  that  depicts  the  direction  the  horizon  is 
pointing  towards. 


pitched  up  or  down.  Pilots  were  asked  to 
remember  the  orientation  of  the  horizon 
line  displayed  on  the  attitude  indicator  and 
the  number  below  the  attitude  indicator  (see 
Figure  lb).  Pilots  first  viewed  a  series  of 
two  attitude  indicators  for  five  trials  and 
progressed  through  a  series  length  of  three 
and  four,  each  containing  five  trials. 


_  e  ,  TWM  „kill  a  domain-specific  piloting  task  similar  to  chess  experiments 
LrW^s«mentTo  asscss  wa! ’developed,  fit  this  task,  pilots  simultaneously  viewed  two 

(e.g.,  Charness,  1  )K\  dc  Cuoo  ,  )  ,  -t  ,,DDCared  on  the  top  half  of  tire  screen,  with  the 

cockpits  for  40  seconds  (see  Anei- 40  seconds  elapsed,  the  computer  presented  a 

second  cockpit  displayed  dncctly  threes  for  3q  seconds,  starting  from  the  presented 

*»  *  -  —  -  - 


pjoure  2  (a)  Meanineful  stalling  flight  situation,  (b) 
Meaningful  future  status  after  a  control  movement  is 
executed. 


Figure  3.  (a)  Nonmcaningful  starting  situation,  (b)  Non- 
meaningful  future  status  (instruments  are  in  conflict  in  both 
cases'). 


In  six  of  the  trials,  the  two  cockpits  were 
related.  That  is,  the  bottom  cockpit 
represented  the  future  state  of  the  aircraft  5  to 
10  seconds  after  applying  one  or  two  control 
movements  to  the  top  cockpit  (see  Figure  2a). 
Three  trials  consisted  of  two  unrelated 
cockpits;  both  depicting  nonmeaningful  flight 
configurations  (see  Figure  3a-b).  Use  of  both 
meaningful  and  nonmeaningful  situations 
allowed  us  to  differentiate  between  retrieval 
originating  from  the  use  of  LTWM  retrieval 
structures  and  retrieval  originating  from  the 
use  of  STM,  respectively.  Because  WM 
capacity  is  thought  to  be  temporary  and 
limited  in  size,  counting  backwards  by  three 
from  a  given  number  requires  processing  and 
storage  that  use  WM  processes,  thus,  3VN4 
capacity  could  not  account  for  pilot  ability  to 
recall  cockpit  information. 

Situation  Awareness.  In  order  to  assess  SA, 
we  developed  trials  consisting  of  a  series  of 
four  screens  that  depicted  a  desired  flight 
status,  a  current  flight  situation  (cockpit  1), 
questions  about  methods  to  achieve  the 
desired  status  (control  movement  selection),  a 
future  flight  situation  intennediate  to  the 
desired  status  (cockpit  2),  and  one  of  two 
types  of  inquiries  about  a  change  in  flight 
status.  Figure  4  provides  the  names  of  each 
screen  and  a  schematic  of  a  trial.  The  first 
screen  contained  a  text  description  of  a 
desired  heading,  altitude,  and  airspeed,  as 
well  as  a  flight  situation.  Pilots  were  asked  to 
assess  the  flight  situation  and  the  desired 
flight  status  specified  in  the  goal  statement, 
and  to  determine  the  flight  control 
movements  required  to  reach  the  goal.  After 
selecting  the  control  movement(s),  pilots 
clicked  “Next”  to  view  the  third  screen 
(cockpit  2).  The  third  screen  depicted  a  future 
control  movements  to  the  starting  situation 


LltT)’ The  SojIS  waTt  o' tain"  ifcoikpTt  2  accurately  depicted  a  ffight  situation  that  would 

^!h  SK^“Ldedfl,a,,ri»,s.ttr 

kDk  2)  would  reach  the  goal,  they  were  asked  a  question  about  the  starting  s, matron  (screen  4a).  If  the 
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a  second  cockpit,  and  determines  it  me  seconu  cu^F. . .  • 

displayed  after  a  consistent  or  inconsistent  judgment,  lespectively 


pilots  decided  that  the  flight  situation  (cockpit  2)  would  not  teach  the  goal,  they  were  asked  to  identify  the 
instruments  inconsistent  with  obtaining  the  goal  (scieen  4b). 


ACCOMPLISHMENTS  (throughout  award  period): 

We  have  counted  a  study  .to  — SS 

individual  dtffeiencesin fltg >  P”  measured  WM  capacity,  the  second  experiment  measured 

^tr»^“^men.  measuted  SA  petfo.mauee.  Detaiis  ahou,  the  design  of  each  of  these 
experiments  can  be  found  in  the  Approach  section  of  this  leport. 

purchase  special  licenses  for  off-site  equipment. 

f  ,,  t  .  frnm  52  tj  S  Navy  student  and  instructor  pilots  (Whiting  Field  in  Milton,  FL)  and  25 

"a, MSU  »7dsI“o.  data  were  oot.ected  by  Ltjg.  F.tolitis,  USN  (NAMRL). 

Data  from  the  WM  capacity,  LTWM  skill,  and  SA  experiments  were  analyzed  as  a  (unction  of  expertise. 
Details  of  the  analyses  can  be  found  in  the  Conclusions  section  of  this  teport. 

CONCLUSIONS: 

indicated  by  previous  research  <e.g„  Soli.  * 


2.0 


o  Novice 


Figure  5.  Sensitivity  (d')  as  a  function  of  LTWM  skill 
for  expert  pilots. 


Percent  of  trials  containing  a  substitution  of  control  movement 

Figure  6.  Relation  of  working  memory  capacity  to  the  percentage 
of  trials  containing  a  substitution  of  a  required  control  movement 
for  novice  pilots. 


oKIp  tn  recall  meaningful  situations  compared  to  novices.  This  is 

LTWM).  One  final  piece  of  evidence  .a  (hat  LTWM skill  praw ed  O  P  ^  ^  ^  L  Ty./M 

capacity  and  LTWM  still  should  have  been  equally 

predictive. 

Given  ,he  evidence  that W« capacity  and 1LT™«1 

examine  die  ™  Th«t  our  icsuhs  tmgges.  dia,  bod, 

™°™p”ciw  a"  d  ^TWM  skid  are  important  to  the  development  of  SA,  but  their  roles  differ  as  a  function 
r^Funher  analyses 

*  »"  Br»  n  Recall  that  Endsle/s  (1995, 2000)  information 

process  acfioii°plans  differently;  expetl  action  planning  necessarily  involves  Level  3  SA  and  novice  action 
planning  does  not. 


S  Novice 


Figure  7.  Accuracy  of  future  judgments  based  on 
selecting  the  correct  versus  incorrect  (adequate 
vs.  inadequate  SA)  controls  as  a  function  of 
expertise. 


In  summary,  it  appears  that  both  WM  capacity 
and  LTWM  skill  have  the  potential  to  identify 
individuals  who  are  more  likely  to  obtain 
adequate  levels  of  SA.  In  addition,  because  WM 
capacity  predicted  novice  action  plan  errors,  it 
may  be  possible  to  facilitate  training  and 
performance  by  designing  systems  that  support 
minimal  WM  demands.  With  respect  to  LTWM 
skill,  it  may  be  possible  to  train  for  better 
organizational  skills.  One  possible  test  is  to  see 
how  well  individuals  organize  common  material. 
People  with  better  organizational  skills  may  be 
more  likely  to  develop  retrieval  structures  that 
will  facilitate  building  and  maintaining  adequate 
SA. 


SIGNIFICANCE: 


In  the  21st  century  Navy  aviators  will  be  asked  to  perform  tasks  in  more  complex  and  dynamically 
chLine  environments  than  ever  before  in  order  to  meet  operational  requirements  (NPRST  Sailor  21 
document  Many “men.  and  future  Navy  aviation  jobs  denraud  the  ability  to  butld  and  matntain  awareness 

of  a  dynamically^ changing  flight  situation  based  on  information  presented  in  visual  cockpit  displays. 

of  a  dynamicalycnang  y comprehend  indications  of  their  aircraft  status,  such  as  airspeed, 

to  Xe  "„d “to  monitor  progress  toward  flight  goals.  The  ability  to  monitor 
instruments  perform  actions,  and  comprehend  a  dynamically  changing  situation  is  required  in  many  avy 
obs  both  ail  and  sea  For  example,  air  traffic  controllers,  tactical  coordinators,  and  submarine  and  ship 
operators  musj  build  and  maintain  awareness  of  the,,  operational  situations.  Thus,  sttuatton  awareness 
abilities  enable  sailors  to  perform  in  abroad  range  of  Navy  jobs. 

Each  of  the  Navy  jobs  described  above  requires  high  situation  awareness  abilities  (Endsley,  1995,  2000), 
and  recent  research  suggests  that  individual  differences  in  cognitive  abilities  pi  edict  flight  situation 
awareness  performance  (e.g.,  Sohn  &  Doane,  2003).  Identifying  recruits  with  the  cognitive  abilities  that 
X  h  gh  s  tuation  awareness  performance  would  facilitate  their  selection  and  classification  into  these 
occupations  and  others  in  which  successful  performance  depends  on  situation  awareness  skills  (Alderton, 

1989;  Sohn  &  Doane). 

The  focus  of  this  research  was  on  developing  new  measures  of  three  components  of 
have  been  shown  to  support  task  performance  on  flight  situation  awareness  tasks  (Sohn  &  Doane  2003), 
capacity  (e.g..  ftaneman  *  Carpenter,  1 980;  Jus,  &  Carpenter,  1992;  Shah  &  Miyake, 
1996)  long-tenn  working  memory  skills  (e.g.,  Ericsson  &  Delaney,  1999;  Ericsson  &  Kmtsch,  1995, 
Kintsch  1998)  and  the  ability  to  reason  about  sequence-dependent  events  or  “event  reasoning  (Doane  & 
Sohn  2001;  Durso  &  Gronlund,  1999).  Computer  administered  tests  were  developed  for  each  Processing 
abilitv  and  were  tested  for  their  predictive  validity  for  flight  situation  awareness  task  performance.  Thus, 
the  first  year  goal  of  providing  new  measures  of  flight  situation  awareness  ablbties  and  relating  them  o 
to  SmTng  and  perfotmauce  was  achieved.  Unfortunately,  because  of  a  lack  of  fimdtng,  the  measures 
were  not  further  developed,  and  the  impact  of  stress  on  these  measures  was  not  examined.  Therefoie,  we 
"So  improve  upon  situation  awareness  ability  assessment,  and  we  were  unable  ,0  provide  new 
ltd  useM  elassif, cation  tools  that  would  benefit  a  broad  spectrum  of  Navy  jobs  that  require  s, .nation 
awareness  abilities. 
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